
United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark Oflici 
Address COMMISSIONER FOR PATENTS -\- 
P.O. Box 1450 

Alexandria, Viiginia 22313-1450 

WWW.USplO.gOV 



APPLICATION NO. 


FILING DATE 


FIRST NAMED INVENTOR 


ATTORNEY DOCKET NO. | 


CONFIRMATION NO. 


09/995,471 


11/28/2001 


Nevenka Dimitrova 


US 010606 


2298 



24737 7590 1 1/29/2004 

PHILIPS INTELLECTUAL PROPERTY & STANDARDS 
P.O. BOX 3001 

BRIARCLIFF MANOR, NY 10510 



EXAMINER 



LANEAU, RONALD 



ART UNIT 



PAPER NUMBER 



3627 

DATE MAILED: 1 1/29/2004 



Please find below and/or attached an Office communication concerning this application or proceeding. 



PTO-90C (Rev. 10/03) 



Office Action Summarv 

WlllWV w H 1 I WUI Iff f f dff If 


Application No. 

09/995,471 


Applicant(s) . 
DIMITROVA ET AL. 


Examiner 

Ronald Laneau 


Art Unit 

3627 





- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 
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Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1)K Responsive to communication(s) filed on 12 August 2004 . 
2a)S This action is FINAL. 2b)Q This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1-15,17-26 and 28-39 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) \Z\ Claim(s) is/are allowed. 

6) E3 Claim(s) 1-15.17-26 and 28-39 is/are rejected. 

7) \Z\ Claim(s) is/are objected to. 

8) n Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)n The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1 .121 (d). 
1 1 )D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-1 52. 
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1 0 Certified copies of the priority documents have been received. 

20 Certified copies of the priority documents have been received in Application No. . 
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application from the International Bureau (PCT Rule 17.2(a)). 
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Response to Amendment 

1 . The amendment filed on 08/12/2004 has been entered. Claims 1-39 remain pending in 
this application. 



Claim Rejections - 35 USC § 102 

2. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

3. Claims 1-15, 17-19, 26 and 28-39 are rejected under 35 U.S.C. 102(b) as being 
anticipated by Wactlar, et al. (5,835,667). 

Wactlar discloses an information tracker (Abstract, Fig. 1; cols. 4-5, lines 30-35) that 
contains a content analyzer with a memory that stores content data received from information of 
machine-sources (video signals, cable and satellite television, audio and radio) an a set of 
machine-readable instructions for analyzing the content data according to query criteria (Figs. 1- 
2 and cols. 4-7 and 11-12); an input device communicatively connected to the content analyzer 
that permits a user to interact with the content analyzer (Figs. 4-5; cols. 1 1-15); and a display 
device communicatively connected to the content analyzer for displaying a result of analysis of 
the content data performed by the content analyzer (Figs. 4-6; cols. 17-18)9 wherein, according 
to the set of machine-readable instructions, the processor of the content analyzer analyzes the 
content data to extract and index one or more stories related to the query criteria (Figs. 1-2.; cols. 
4-7) and wherein, the processor of the content analyzer uses the query criteria to spot a subject in 
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the content data, extract one or more stories from the content data, resolve the inference names in 
the extracted one or more stories, and display a link to the extracted one or more stories on the 
display device (Figs. 1-2,* col. 9, cols. 17-18) and wherein, in addition to displaying the link to 
the extracted one or more stories, the content information about the subject to display one or 
more links to a shopping web-site, such that the user can purchase goods related to the subject 
(Figs. 1-2; col. 9. cols. 17-18 - sale to users). 

Wactlar discloses that the names in the extracted stories are resolved and inferenced 
using ontology (cols. 4-5, col.. 7 and col. %. 

Wactlar discloses that if more that one store is extracted the processor indexes the stories 
according to name, topic and keyword (cols. 4-5, and 7-1 1). 

Wactlar discloses that the stories are further ordered based on causality relationship 
and/or temporal relation ship and/or alphabetically (cols. 7-1 1). 

Wactlar discloses that the query criteria includes a request input by the user through the 
input device and the processor analyzes the content data according to the request and a user 
profile, which includes information about the user's interests, which can be updated by 
integrating information in the request with existing information in the user profile (cols. 14-15 
and 17-183). 

Wactlar discloses that the content analyzer comprises a knowledge base, which includes a 
plurality of known relationships, and the processor analyzes the content data according to the 
knowledge base. (cols. 4-5). 

Wactlar discloses that the known relationships include maps of a known face and voice to 
a name and a name to related information (Fig. 6,* cols. 11-13, cols. 17-18 - Find video with 
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same person - maps known face and voice to name and related information thus increasing 
reliability). 

Claim 15 is rejected for the same reasons set forth above in claims 9, and 12-13. 

Wactlar discloses a person spotting function that matches known faces to extracted faces, 
known voices to extracted voices, scans the extracted text to match known names and calculates 
a probability of a particular person being present (Fig.6; cols. 11-13, cols. 17-18). 

Wactlar discloses that the content analyzer is communicatively connected to a second 
information source for providing access to additional content data, the additional content data 
being analyzed for relevant stories (Fig. 1 - Extra Footage, New Video Footage; col. 6 - it is 
preferable that the raw video material incorporates not only television footage but also the 
unedited source materials, shown generally as extra footage - raw material could also include 
pure text, audio only or video only). 

Method claims 28-36 and 38 are rejected for the same reasons set forth above. 

Wactlar discloses that indexing occurs according to predetermined criteria, extracting a 
causality relationship, and extracting a temporal relationship, calculating a rating for each of the 
extracted stories from one or more characteristics of the extracted stories and prioritizing the 
extracted stories (cols. 4-5, cols. 7-1 1). 

Wactlar discloses creating a hyperlinked index (cols. 4-5 and throughout - discloses that 
the system is utilized on-line and on the Internet and thus would be hyperlinked). 

Wactlar discloses that the content analyzer is centrally located and the user accesses the 
content analyzer via a communications network (cols. 4-5 and throughout - discloses that the 
system is on-line and on the Internet). 
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Information tracing retrieval system claim 39 is rejected for the reasons set forth above. 

Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 1 02 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

5. Claims 20 and 21 are rejected under 35 U.S.C. 103(a) ms being unpatentable over 
Elenbams, et al. (W0 00/39707). 

Elenbaas discloses as set forth above. The examiner directs attention to the fact that 
Elenbaas further discloses that the system is utilized with video broadcasts that contain an audio 
signal and that these signals are utilized in the classification process and that it can be utilized for 
classification from sources including the Internet and the World Wide Web, which contain radio 
signals, i.e. Real Player, as well. Thus, the substitution/utilization of an audio signal for content 
data and a radio station as the information source would have been obvious to one of ordinary 
skill in the art at the time of the invention as it is irrelevant whether the content data/information 
source is that of a radio or television provider as each is a standard for providing part of the ever 
increasing supply of information and entertainment options in the broadcast media, even as set 
forth in the background of the Description of Related Art (pp. 1-2) in the multimedia venues and 
would have provided the same types of information to the content analyzer of claim 1. 
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6. Claims 22-25 are rejected under 35 U.S.C. 103(a) as being unpatentable over Elenbaas, et 
al. (WO 00/39707) and/or Wactlar, et al. (5,835,667) in view of Mei Kobayashi and Koichi 
Takea, Information Retrieval on the Web, AMC Computing Surveys, Vol. 32, No. 2, June 2000, 
pp. 144-173 and further in view of Carol Tenopir, Online Databases - Trends for the Next Five 
Years, Library Journal, Vol. 125, No. 16, October 1, 2000, pp. 38 and 40. 

Elenbaas and Wactlar disclose as set forth above. However, neither Elenbaas nor Wactlar 
(assuming arguendo) specifically disclose that the content analyzer is communicatively 
connected to a second information source for providing access to additional content data, the 
additional content data being analyzed for relevant stories. Kobayashi discloses that Dialog, 
Lexis-Nexis, Yahoo and Altavista are all communicatively coupled via their search engines to 
databases of images, i.e. a second information source that is analyzed for relevant stories (pp. 
164-165). Kobayshi discloses that internet retrieval of multimedia resources in addition to 
regular web based resources is a potentially lucrative application and that the vast majority of 
publication support this view and there are many talented scientists working towards 
implementing more concrete solutions and more efficient and user- friendly solutions (pp. 165- 
166). Tenopir further teaches that the online industry sees a need and desirability to link and 
index all types of databases whether text based or multimedia as the demand for information 
grows and the need to add value to information products and services should be on the mind of 
any information intermediary, whether from libraries or database companies (pp. 38 and 4%. 
Thus, it would have been obvious to one of ordinary skill in the art at the time of the invention to 
have utilized a content analyzer with a the Internet, whether web site or image or video, to obtain 
additional stories/content as taught in Kobayslli and Tenopir for the explicit reasons set forth. 
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Elenbaas and Wactlar disclose as set forth above. However, they do not disclose either a 
first or second approach or finding intersection/matching stories. The examiner notes that 
scanning a web site involves using a "robot" Known as a "spider' or crawler" which feeds data 
into an index and utilizes software to then sort and rank indexed information and then a query is 
performed by Boolean (keyword), natural language, etc. However, as se forth above, Kobayashi 
discloses that Yahoo and Altavista are all communicatively coupled via their search engines to 
databases of images, i.e. a second information source that is analyzed for relevant stories (pp. 
164-165). Kobayshi discloses that internet retrieval of multimedia resources in addition to 
regular web based resources is a potentially lucrative application and that the vast majority of 
publication support this view and there are many talented scientists working towards 
implementing more concrete solutions and more efficient and user-friendly solutions (pp. 165- 
166). Tenopir further teaches that the online industry sees a need and desirability to link and 
index all types of databases whether text based or multimedia as the demand for information 
grows and the need to add value to information products and services should be on the mind of 
any information intermediary, whether from libraries or database companies (pp. 38 and 40). 
Thus, it would have been obvious to one of ordinary skill in the art at the time of the invention to 
have a first and second approach ms the search engines already utilize both in conjunction with 
each other as internet portals to obtain the content, whether web site or image or video, to obtain 
additional stories/content as taught in Kobayshi and Tenopir for the explicit reasons set forth. 
Further it is inherent that Altavista performs the matching and intersection function is performed. 
See Altavista Asserts Search Strength With Web's largest index and Directory, PR Newswire, 
May 4, 2000, pg. 1 - duplicate and near-duplicates pages are weeded out of the result set, i.e. 
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matching and intersecting stories that are retrieved as a result of both the first and second 
approaches found in the additional data content are compared and removed. 

Response to Arguments 

7. Applicant's arguments filed 08/12/2004 have been fully considered but they are not 
persuasive. 

Applicant has requested the status of claims 28 and 37. In response to applicant's 
request, the status of claims 28 and 37 were clearly defined in the office action summary which 
indicated that claims 1-39 have been rejected. That includes claims 28 and 37. In regards to 
Wactlar, applicant argues that Wactlar fails to anticipate extracting "faces, speech, and text" 
from content data, making a "first match of known faces to the extracted faces," making a 
"second match of known voices to the extracted speech," scanning the "extracted text to make a 
third match to known names," and calculating a "probability of a particular person being present 
...based on the first, second, and third matches." Contrary to applicant's arguments, figure 6 of 
Wactlar appears to be doing all of these when it comes to matching faces, speech, and text as 
seen. Wactlar' s system implicitly would have to calculate the probability of finding a face at the 
end during processing. As far as applicant's arguments about no suggestion or motivation to 
combine the references, the examiner recognizes that obviousness can only be established by 
combining or modifying the teachings of the prior art to produce the claimed invention where 
there is some teaching, suggestion, or motivation to do so found either in the references 
themselves or in the knowledge generally available to one of ordinary skill in the art. Se In re 
Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 1988) and In re Jones, 958 F.2d 347, 21 
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USPQ2d 1941 (Fed. Cir. 1992). Applicant's arguments are deemed unpersuasive and claims 1 
and 3-16 remain rejected and made final. 

Conclusion 

8. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 

9. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Ronald Laneau whose telephone number is (703) 305-3973. The 
examiner can normally be reached on Mon-Fri from 8:30am - 6:00pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Robert Olszewski can be reached on (703) 308-5183. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. . 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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Examiner 
Art Unit 3627 




